Online and adaptive anomaly Detection: detecting intrusions in unlabelled audit data streams

Intrusion detection has become a widely studied topic in computer security in recent years. Anomaly detection is an intensive focus in intrusion detection research because of its capability of detecting unknown attacks. Current anomaly IDSs (Intrusion Detection System) have some difficulties for practical use. First, a large amount of precisely labeled data is very difficult to obtain in practical network environments. In contrast, many existing anomaly detection approaches need precisely labeled data to train the detection model. Second, data for intrusion detection is typically steaming and the detection models should be frequently updated with new incoming labeled data. However, many existing anomaly detection methods involve off-line learning, where data is collected, manually labeled and then fed to a learning method to construct normal or attack models. Third, many current anomaly detection approaches assume that the data distribution is stationary and the model is static accordingly. In practice, however, data involved in current network environments evolves continuously. An effective anomaly detection method, therefore, should have adaptive capability to deal with the “concept drift” problem while effectively detects intrusions in unlabelled audit data streams.